Systems and methods for deblocking filtering

ABSTRACT

Methods and systems for in-loop filtering may comprise receiving a video comprising at least one frame. The frame may comprise at least one block of pixels. A content complexity of the block of pixels may be determined. A viewing characteristic of the video may be determined. A number of borders to be filtered may be determined based on at least one of the content complexity or the viewing characteristic. A deblocking filter strength may be determined based on at least one of the content complexity or the viewing characteristic. The number of borders of the block of pixels may be filtered according to the deblocking filter strength.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/699,588, filed Jul. 17, 2018, the contents of which isincorporated herein by reference in its entirety.

BACKGROUND

In block-based video coding, each frame may have blockiness and/or othercoding artifacts. In order to reduce or remove such artifacts,deblocking filtering may be applied. A deblocking filter may haveparameters. The deblocking filter parameters may be determined based ona quantization parameter (QP).

SUMMARY

Deblocking filtering may comprise using a deblocking filter havingparameters that are adaptively determined. The parameters may comprise anumber of borders of a block of pixels of a frame of a video to befiltered and a filter strength. The parameters may be adaptivelydetermined for the block based on perceptual and/or statisticalproperties of the block and/or the video. For example, the number ofborders to be filtered and/or the filter strength may be determinedbased on a content complexity of the block, such as a texture shown inthe block and/or features shown in the block. The number of borders tobe filtered and/or the filter strength may be determined based on aviewing characteristic of the video, such as a characteristic of anintended display for viewing the video or a characteristic of anintended setting for viewing the video. The determined number of bordersof the block may be filtered using the determined filter strength.

BRIEF DESCRIPTION OF THE DRAWINGS

The following drawings show generally, by way of example, but not by wayof limitation, various examples discussed in the present disclosure. Inthe drawings:

FIG. 1 shows a diagram of an example method.

FIG. 2 shows example blocks of pixels.

FIG. 3 shows example blocks of pixels.

FIG. 4 shows an example deblocking filtering parameter graph.

FIG. 5 shows an example deblocking filtering parameter graph.

FIG. 6 shows a flow diagram of an example method.

FIG. 7 shows a flow diagram of an example method.

FIG. 8 shows a flow diagram of an example method.

FIG. 9 shows an example computing environment.

DETAILED DESCRIPTION

Block-based video coding methods may result in frames having blockinessand/or other coding artifacts. In order to reduce or remove suchartifacts, deblocking filtering may be applied. A deblocking filter mayhave parameters. The parameters may comprise a number of borders of ablock of pixels of a frame of a video to be filtered and/or a filterstrength, as examples.

The parameters may be adaptively determined. As an example, theparameters may be determined for a block of pixels of a frame of avideo, for the frame, and/or for the video based on perceptual and/orstatistical properties of the block, the frame, and/or the video. Theparameters may be determined based on a content complexity of the block,the frame, and/or the video, such as a texture shown in the block and/orfeatures shown in the block, the frame, and/or the video. The parametersmay be determined based on a viewing characteristic of the video, suchas a characteristic of an intended display for viewing the video or acharacteristic of an intended setting for viewing the video. Thedetermined number of borders of the block may be filtered using thedetermined filter strength.

FIG. 1 shows an example coding method 100. The coding method 100 may beperformed by an encoder. The coding method 100 may be performed by acodec. The coding method 100 may comprise a method that conforms to theITU-T H.264/ISO/IEC 14496-10 (AVC) standard or the ITU-T H.265/ISO/IEC23008-2 (HEVC) standard. The coding method 100 may comprise a lossyvideo coding method.

The coding method 100 may comprise receiving a video 101. The video 101may comprise one or more frames. The frames may show pictures. Theframes may comprise pixels. Each of the pixels may have a frequencyand/or wavelength that is perceived by the Human Visual System as color.The color may be represented as chroma values in an RGB color space. Thecolor may be represented as chroma values in a YUV color space. Chromavalues in the RGB color space may be converted to chroma values in theYUV color space.

One or more of the frames may be split into blocks of pixels, such asrectangular blocks of pixels. The frame may be split into macroblocks.For example, the frame may be split into 16×16 pixels macroblocks, inaccordance with the ITU-T H.264 and ISO/IEC 14496-10 (Advanced VideoCoding (AVC)) standard. The size of the blocks may be based on a numberof details shown in the frame. For example, an area of the frame showingmore details may be split into smaller blocks than another area of theframe showing fewer details. An indication of partitioning of the blocksof pixels to may be sent to a decoder.

A chrominance and/or luminance of a block may be determined by samplingpixels of the block. For example, one pixel of a block may be sampled todetermine chrominance of the block. Several pixels of the block may besampled to determine luminance of the block. Other sampling formats maybe used.

One or more of the frames may be encoded by intra-picture prediction.Intra-picture prediction may comprise predicting data spatially fromblock-to-block within the frame. Intra-picture prediction may not dependon other frames of the video.

One or more of the frames may be encoded by inter-picture prediction.Inter-picture prediction may comprise determining motion data. Themotion data may be determined for based on the determined chrominanceand/or luminance of the blocks. The motion data may comprise a referenceframe and a motion vector indicating a pixel of the reference frame witha similar chrominance and/or luminance to one of the sample pixels. Themotion data may be sent to the decoder. Mode decision data associatedwith the inter-picture prediction may be sent to the decoder. Thedecoder may perform inter-picture prediction using the motion data andmode decision data.

A residual signal may comprise the difference between a block and acorresponding prediction. The residual signal may be transformed from aspatial domain to a frequency domain, such as by discrete cosinetransform (DCT), another linear spatial transform, or integer transformproposed by the ITU-T H.264 and ISO/IEC 14496-10 (AVC), or the ITU-TH.265 and ISO/IEC 23008-2 (High-Efficiency Video Coding (HEVC)). Thetransform may remove redundancy in the data describing intensity of thepixels of the blocks. The transformation may yield a matrix ofcoefficients for each block. The coefficients may represent spatialfrequencies within the block.

The coefficients may be quantized. Quantization may comprise scaling thecoefficients by a quantization parameter (QP). For example, quantizationmay comprise dividing the coefficients by the QP. Quantization maycomprise dividing the matrix of coefficients by a matrix of QP's.Quantization may yield a quantized matrix for each block. The quantizedmatrix may comprise less information and, thus, require less bits toencode. The quantized matrices may be sent to a decoder.

The QP value used may vary across blocks of pixels of a frame. Thedifference in QP values may be constrained among neighboring blocks. Thevalue of the QP may determine the size of a quantization step (e.g., thecoarseness of rounding of the coefficients). Quantization with a higherQP value may result in higher compression. However, a higher QP valuemay result in more information discarded during quantization, such asloss in precision of frequencies and the removal of frequenciesrepresented by coefficients rounded to zero. Therefore, quantizationwith a higher QP may alter the appearance of fine textures shown in theblock, resulting in blocking (e.g., “blockiness”). The blockiness may beperceptible to the Human Visual System. Quantization with a lower QPvalue may result in a loss of frequencies that may be imperceptible tothe Human Visual System. Yet, quantization with a lower QP may result inmore information being retained and lower compression.

The quantized matrices may be entropy encoded. A decoder processing loopmay be performed. Based on the encoder performing the decoder processingloop, the encoder and the decoder may generate identical predictions forsubsequent data. The decoder processing loop may comprise inversescaling and inverse transforming the quantized transform coefficients todetermine the decoded approximation of the residual signal. The residualsignal may be added to the motion data.

The result of the residual signal added to the motion data may be fed toone or more in-loop filters, such as a deblocking filtering and/orsample adaptive offset (SAO). The in-loop filters may remove artifactscaused by the block-wise processing and/or quantization. The in-loopfilters may smooth edges of features shown in the reconstructed picture.The in-loop filters may produce a final picture representation. Thefinal picture representation may comprise a duplicate of the output ofthe decoder. The final picture representation may be stored in a decodedpicture buffer. The final picture representation may be used for theprediction of subsequent frames.

To reduce blockiness, such as from block-wise processing and/orquantization, deblocking filtering may be performed. The deblockingfiltering may conform to ITU-T H.264 and ISO/IEC 14496-10 (AVC), orITU-T H.265 and ISO/IEC 23008-2 (HEVC). Deblocking filtering maycomprise applying a deblocking filter to a reconstructed picture.Deblocking filtering may improve objective and/or subjective quality ofa reconstructed picture, such as quality as perceived by the HumanVisual System (HVS). Deblocking filtering may enhance continuity ofblock boundaries (e.g., borders, edges, etc.). For example, deblockingfiltering may remove coding artifacts and/or increase coding gain.Deblocking filtering, when applied to a frame used as a reference framefor motion-compensated prediction of subsequent frame, may result in anincrease in coding efficiency and, thus, bitrate savings.

A decoder may perform filtering decisions and/or operations insynchronization with the encoder. As a result, deblocking filteringand/or other in-loop filtering operations may ensure a level of qualityof the picture that is output, such as across decoders of differentmanufacturers. However, if the deblocking filter is too strong, thedeblocking filtering may result in blurring of features in the blocks.

The deblocking filter may have one or more parameters, depending on aparticular codec. For example, it may have a β threshold parameter and at_(c) threshold parameter. The β threshold parameter may be used todetermine a number of vertical and/or horizontal block borders to befiltered. The t_(c) threshold parameter may be used to determine afilter strength. The t_(c) threshold parameter may be used to determinea number of pixels from the block borders to be modified. The t_(c)threshold parameter may be used to determine a maximum absolute value ofmodifications allowed for pixel values. The maximum absolute value ofmodifications may limit blurriness caused by the deblocking filtering.The determined number of block borders may be filtered using thedetermined filter strength. For example, the determined number of pixelsfrom the block borders may be modified for the determined number ofvertical and/or horizontal block borders.

Currently, the β threshold parameter and the t_(c) threshold parametermay be determined by a human operator. The human operator may determinethe β threshold parameter and a t_(c) threshold parameter arbitrarily.The human operator may determine the β threshold parameter and a t_(c)threshold parameter based on a reference. Currently, a same β thresholdparameter and a same t_(c) threshold parameter may be determined fordifferent blocks of pixels of a frame, for different frames of a video,and/or for a complete workflow for a large amount of sequences.

According to the H.265/MPEG-HEVC standard, the β threshold parameter andthe t_(c) threshold parameter may be determined based on the value ofthe QP used for quantization. The β threshold parameter and the t_(c)threshold parameter may be determined based on the value of the QP usedfor quantization because blocking artifacts may be the consequence ofblock transform and quantization. The β threshold parameter and thet_(c) threshold parameter may be determined for a block of pixels basedon a QP value used for neighboring blocks having a common border withthe block. For example, the β threshold parameter and the t_(c)threshold parameter may be determined based on an average QP value oftwo neighboring blocks having a common border. Improvements in theefficiency of determining the β threshold parameter and the t_(c)threshold parameter, such as for high resolution (UHD), high bit depth,and/or high dynamic range (HDR) video, are needed.

FIG. 2 shows neighboring blocks of pixels of a frame of a video, Block Pand Block Q. Block P and Block Q may be separated by a block boundary.Sample pixels p0, p1, p2, p3 in Block P may correspond to sample pixelsq0, q1, q2, q3 in Block Q. As shown in FIG. 2, there may be variancebetween sample pixels in Block P p0, p1, p2, p3 and the correspondingsample pixels q0, q1, q2, q3 in Block Q, such as a variance in lumavalue. The variance may result from quantization, block-basedtransformation, and/or motion compensation of the blocks. The variancein values of pixels in Block P near the block boundary and pixels inBlock Q near the block boundary may result in visually noticeablediscontinuities along boundaries of blocks. The variance may result inblocking artifacts at the block boundaries. The blocking artifacts maybe perceptible to the Human Visual System. The blocking artifacts maydecrease the overall picture quality of the frame. As shown in FIG. 2,relatively low spatial activity on opposite sides of the block boundarymay indicate the blocking artifacts and/or a discontinuity at the blockboundary. Quantization using a larger QP may result in greaterdistortion of boundaries, as more data may be lost. Motion-compensatedprediction may result in discontinuities in the interior of blocks.Deblocking filtering may be performed to reduce the discontinuitiesand/or smooth the boundaries.

FIG. 3 shows example blocks of pixels of a frame of a video. Deblockingfiltering may be performed on the frame. The deblocking filtering may beperformed on non-overlapping blocks of pixels. The deblocking filteringmay be performed on 4×4 pixel blocks positioned on a grid splitting theframe into 4×4 pixel samples, such as in conformity with theH.264/MPEG-AVC standard. As shown in FIG. 3, the deblocking filteringmay be performed on 8×8 pixel blocks positioned on a grid splitting theframe into 8×8 pixel samples, such as in conformity the H.265/MPEG-HEVCstandard. Deblocking filtering 8×8 pixel blocks, as opposed to 4×4 pixelblocks, for example, may reduce computational complexity. However,deblocking filtering 8×8 pixel blocks, as opposed to 4×4 pixel blocks,may result in reduced overall picture quality. As a result, picturequality may trade-off with computational simplicity. Deblockingfiltering may be performed on 4×4 pixel blocks and 8×8 pixel blocks. Forexample, 4×4 pixel blocks may be used in areas with relatively highspatial and/or temporal activity. The blocks may be deblocked inparallel.

FIG. 4 shows an example relationship between QP values and t_(c)threshold parameter values. At low QP values, the t_(c) thresholdparameter may have a value of 0. Above a particular QP value, the valueof the QP and the value of the t_(c) threshold parameter may be directlyproportional. For example, the t_(c) threshold parameter may have avalue of 0 for QP values less than or equal to a particular value, suchas 15. The t_(c) threshold parameter having a value of 0 for QP valuesless than or equal to a particular value may disable deblocking and/ormay reduce computational resources used for in-loop filtering. For QPvalues greater than 15, a greater QP value may result in a greater valueof the t_(c) threshold parameter. As a result of the t_(c) thresholdparameter increasing directly with the QP value, more deblocking may beperformed based on use of a greater QP value.

FIG. 5 shows an example relationship between QP values and β thresholdparameter values. At low QP values, the β threshold parameter may have avalue of 0. Above a particular QP value, the value of the QP and thevalue of the β threshold parameter may be directly proportional. Forexample, the β threshold parameter may have a value of 0 for QP valuesless than or equal to 15. For QP values greater than 15, a greater QPvalue may result in a greater value of the β threshold parameter.

Methods are needed for adaptively and efficiently deblocking filtering.The methods may comprise adaptively determining the β thresholdparameter and/or the t_(c) threshold parameter. The methods may providea low computational complexity deblocking filter, such as for expeditedcoding.

The methods may comprise determining the β threshold parameter and/orthe t_(c) threshold parameter of a deblocking filter for a block ofpixels based on a complexity of the block of pixels. The complexity ofthe block may comprise a complexity of a feature or texture shown in theblock. For example, a higher t_(c) threshold parameter may be determinedfor a block showing a flat area, such as a blue sky or white clouds,than a block showing a relatively complex texture, such as a tree withleaves, a forest, or sparkling water. For example, a higher β thresholdparameter may be determined for a block showing a flat area than a blockshowing a relatively complex texture. The β threshold parameter and/orthe t_(c) threshold parameter for a block may be determined based on theβ threshold parameter and/or the t_(c) threshold parameter previouslydetermined for neighboring blocks, such as in order to minimizevariations in deblocking filter strength applied to neighboring blocksand, as a result, observed by the Human Visual System.

The complexity of a block may be determined based on a number of detailsor features shown in the block. The complexity of a block may bedetermined based on variation of pixels in the block. The complexity ofa block may be determined according to any of several methods.

The methods may comprise determining the β threshold parameter and/orthe t_(c) threshold parameter based on a desired perceptual visualquality of the video, such as displayed to an end-user and as perceivedby the Human Visual System. The β threshold parameter and/or the t_(c)threshold parameter may be determined based on one or more viewingand/or perceptual characteristics associated with the desired perceptualvisual quality of the video. The viewing characteristic may comprise anintended distance between a viewer of the video and a display of thevideo. For example, a lesser deblocking filter strength and/or lessborders may be filtered based on a greater distance, as blockingartifacts may be less noticeable at greater distances from a display.The viewing characteristic may comprise a resolution of the video. Theviewing characteristic may comprise a display width of the video. Theviewing characteristic may comprise a viewing angle of the video. Theviewing characteristic may comprise a display pixel density of thevideo. The viewing characteristic may comprise an intended and/orexpected ambient illumination of a viewing setting of the video. Theviewing characteristic may comprise a pixel dimension of a sinusoidalgrating cycle on the display of the video. Determining the β thresholdparameter and/or the t_(c) threshold parameter based on the viewingcharacteristic may minimize the use of in-loop filter computationalcomplexity resources and/or may expedite the coding process byminimizing the number of pixels processed. Also, determining the βthreshold parameter and/or the t_(c) threshold parameter may be doneaccording to a Contrast Sensitivity Function (CSF). The CSF may be basedon a Human Visual System (HVS) nonlinear transformation model followedby a Modulation Transfer Function (MTF). Also, the CSF may be generated,for example, in accordance with one or more of the following: Barten(Ramp) threshold and Schreiber threshold.

Determining the β threshold parameter and/or the t_(c) thresholdparameter may comprise determining an offset. The offset may be dividedby a value. For example, the offset may be divided by 2. The result ofthe offset divided by the value may be added to the QP value used. Theresult of the offset divided by the value added to the QP value may beused to determine β threshold parameter and/or the t_(c) thresholdparameter, such as by determining the corresponding β thresholdparameter and/or the t_(c) threshold parameter using the table/graph inFIG. 3 and/or the table/graph in FIG. 4. The offset may be determinedadaptively. The offset may vary according to the texture and/or contentcomplexity of one or more blocks of pixels of a frame. As a result, thedeblocking filter strength and/or the number of borders to be filteredmay be adapted based on video content properties.

A hierarchical coding structure may be used to quantize and/ordeblocking filter one or more frames of a video. The hierarchical codingstructure may comprise using QP cascading. The QP cascading may increaseoverall coding gain. The hierarchical coding structure may comprisecoding different hierarchical levels using different QP values. Forexample, lower QP values may be used for lower hierarchical levels andhigher QP values may be used for higher hierarchical levels. Thehierarchical coding structure may comprise using a stronger deblockingfilter strength for hierarchical levels coded with higher QP values,such as to remove coding artifacts resulting from the use of higher QPvalues and/or coarser coding mode decisions.

An a posteriori deblocking filter adaptation may comprise deblockingfiltering applied to a block after coding the block. Deblockingfiltering may be applied to the block after coding the block and blocksimmediately neighboring the block. Blockiness of the encoded block maybe determined. For example, determining the blockiness may comprisedetermining pixel differences across borders of the block. Determiningthe blockiness may comprise comparing differences between border pixelsof neighboring blocks to differences between adjacent pixels within eachblock. The a posteriori deblocking filter adaptation may be applied to ablock after coding the block as a second pass in a two-pass file codingprocess. The a posteriori deblocking filter adaptation may be applied toa block as a same pass frame coding process.

For example, the a posteriori deblocking filter adaptation may beperformed after all blocks neighboring a block have been encoded. For ablock B(i,j), the neighboring blocks may comprise blocks B(i−1, j),B(i+1, j), B(i,j−1), and B(i,j+1). The encoder may make regular codingdecisions. The encoder my update the reconstructed frame. The encodermay store information sufficient to encode a block in memory. Theinformation may comprise partition/quadtree information, motion vectors,and residuals, as examples. The encoder may not apply entropy coding(e.g., Context-Adaptive Binary Arithmetic Coding (CABAC)) after codingthe blocks and/or before the a posteriori deblocking filter adaptation.After neighboring blocks are encoded, the a posteriori deblocking filteradaptation may be performed. Entropy coding may be performed after the aposteriori deblocking filter adaptation.

The described methods and systems for adaptive deblocking filtering maybe used to optimally remove blocking artifacts from frames of a videowithout post-processing the video. The described methods and systems foradaptive deblocking filtering may improve visual quality of a deblockedvideo while reducing computational complexity.

FIG. 6 shows an example method 600. At step 610, a video may bereceived. The video may be received by an encoder. The video may bereceived by a codec. The video may be received by a decoder. The videomay comprise Standard Dynamic Range (SDR) video. The video may compriseHigh Dynamic Range (HDR) video. The video may comprise high depth video,such as 10 bit depth video, or low bit depth video. The video maycomprise high or standard dynamic range video. The video may comprisehigh resolution video, such as 4K resolution video having a resolutionof 3840×2160 pixels, or low resolution video. The method 600 may be moreefficient for higher resolution video, higher bit depth video, andhigher dynamic range video. The video may comprise one or more frames. Aframe of the video may comprise a plurality of pixels. The pixels may bedivided into blocks, such as 4×4 pixel blocks, 8×8 pixel blocks, 16×16pixel blocks and the like.

At step 620, a content complexity of one or more blocks of pixels of theframe of the video may be determined. The complexity of the block maycomprise a complexity of a feature or texture shown in the block. Forexample, a block showing a flat area may have a lower complexity than ablock showing a relatively complex texture. The complexity of a blockmay be determined based on a number of details or features shown in theblock. The complexity of the block may be determined based on variationof pixels in the block. The complexity of the block may be determinedaccording to any of several methods.

At step 630, a number of borders of the block of pixels may bedetermined. The number of borders of the block of pixels may bedetermined based on a β threshold parameter. The β threshold parametermay be determined based on the content complexity. For example, a higherβ threshold parameter may be determined for a block showing a flat areathan a block showing a relatively complex texture.

Determining the β threshold parameter may comprise determining anoffset. The offset may be divided by a value. For example, the offsetmay be divided by 2. The result of the offset divided by the value maybe added to the QP value used. The result of the offset divided by thevalue added to the QP value may be used to determine β thresholdparameter, such as by determining the corresponding β thresholdparameter using the table/graph in FIG. 4. The offset may be determinedadaptively. The offset may vary according to the texture and/or contentcomplexity of one or more blocks of pixels of a frame.

At step 640, a deblocking filter strength may be determined. Thedeblocking filter strength may be determined based on a t_(c) thresholdparameter. The t_(c) threshold parameter may be determined based on thecontent complexity. For example, a higher t_(c) threshold parameter maybe determined for a block showing a flat area than a block showing arelatively complex texture.

Determining the t_(c) threshold parameter may comprise determining anoffset. The offset may be divided by a value. For example, the offsetmay be divided by 2. The result of the offset divided by the value maybe added to the QP value used. The result of the offset divided by thevalue added to the QP value may be used to determine t_(c) thresholdparameter, such as by determining the corresponding t_(c) thresholdparameter using the table/graph in FIG. 3. The offset may be determinedadaptively. The offset may vary according to the texture and/or contentcomplexity of one or more blocks of pixels of a frame.

At step 650, the number of borders may be filtered using the deblockingfilter strength. At step 660, the video may be caused to be output. Theblock having the filtered borders may be caused to be output. A framehaving the block having the filtered borders may be caused to be output.The video may be caused to be output by sending the video, such as bystreaming the video. The video may be sent to a user device, a displaydevice, a mobile device, and/or another computing device, as examples.The video may be caused to be output by displaying, such as via adisplay. The video may be caused to be output by storing the video, suchas on an external storage device, a user device, a portable device, acloud device, and/or a portion of a partitioned memory.

Steps 610-650 may be performed for a plurality of blocks in the frame.Method 600 may be performed as a single pass video frame codingworkflow. Method 600 may be performed as a double-pass video framecoding workflow. Method 600 may be incorporated in an existing or newvideo codec standard.

For example, an encoder may receive a video. The video may compriseframes showing a tree with leaves blowing in the wind against a backdropof a clear sky. The encoder may determine a content complexity of ablock of pixels of one of the frames showing a portion of the leaves.The encoder may determine a content complexity of a block of pixels ofthe frame showing the clear sky. The encoder may determine a highercontent complexity of the block of pixels showing the leaves than theblock of pixels showing the clear sky.

Based on the content complexities, the encoder may determine a number ofborders of each of the frames to be deblocking filtered. Based on thecontent complexities, the encoder may determine a deblocking filterstrength for each of the blocks. The encoder may determine thedeblocking filter strength for each of the blocks based on deblockingfilter strengths determine for adjoining blocks. For example, theencoder may determine a deblocking filter strength for the block showingthe portion of the leaves based, in part, on another block showing theportion of the leaves adjacent to the block. The encoder may filter thedetermined number of borders of each of the blocks using the deblockingfilter strengths determined for each of the blocks.

FIG. 7 shows an example method 700. At step 610, a video may bereceived. The video may be received by an encoder. The video may bereceived by a codec. The video may be received by a decoder. The videomay comprise one or more frames. A frame of the video may comprise aplurality of pixels. The pixels may be divided into blocks, such as 4×4pixel blocks or 8×8 pixel blocks.

At step 720, a viewing characteristic of the video may be determined.The viewing characteristic may comprise an intended distance between aviewer of the video and a display outputting the video. The viewingcharacteristic may comprise a resolution of the video. The viewingcharacteristic may comprise a display width of the video. The viewingcharacteristic may comprise a viewing angle of the video. The viewingcharacteristic may comprise a display pixel density of the video. Theviewing characteristic may comprise an intended and/or expected ambientillumination of a viewing setting of the video. The viewingcharacteristic may comprise a pixel dimension of a sinusoidal gratingcycle on the display of the video. The viewing characteristic maycomprise a reflection coefficient of the SDR/HDR video display.

At step 730, a number of borders of the block of pixels may bedetermined. The number of borders of the block of pixels may bedetermined based on a β threshold parameter. The β threshold parametermay be determined based on the viewing characteristic. For example, ahigher β threshold parameter may be determined for a video that isintended to be viewed at a larger viewing distance. For example, asmaller β threshold parameter may be determined for a video that isintended to be viewed on a device with a wider display width.

Determining the β threshold parameter may comprise determining anoffset. The offset may be divided by a value. For example, the offsetmay be divided by 2. The result of the offset divided by the value maybe added to the QP value used. The result of the offset divided by thevalue added to the QP value may be used to determine β thresholdparameter, such as by determining the corresponding β thresholdparameter using the table/graph in FIG. 4. The offset may be determinedadaptively. The offset may vary according to the texture and/or contentcomplexity of one or more blocks of pixels of a frame.

At step 740, a deblocking filter strength may be determined. Thedeblocking filter strength may be determined based on a t_(c) thresholdparameter. The t_(c) threshold parameter may be determined based on theviewing characteristic. For example, a lower t_(c) threshold parametermay be determined for a video that is intended to be viewed in lowerambient lighting level.

Determining the t_(c) threshold parameter may comprise determining anoffset. The offset may be divided by a value. For example, the offsetmay be divided by 2. The result of the offset divided by the value maybe added to the QP value used. The result of the offset divided by thevalue added to the QP value may be used to determine t_(c) thresholdparameter, such as by determining the corresponding t_(c) thresholdparameter using the table/graph in FIG. 3. The offset may be determinedadaptively. The offset may vary according to the texture and/or contentcomplexity of one or more blocks of pixels of a frame.

At step 750, the number of borders may be filtered using the deblockingfilter strength. At step 760, the video may be caused to be output. Theblock having the filtered borders may be caused to be output. A framehaving the block having the filtered borders may be caused to be output.The video may be caused to be output by sending the video, such as bystreaming the video. The video may be sent to a user device, a displaydevice, a mobile device, and/or another computing device, as examples.The video may be caused to be output by displaying, such as via adisplay. The video may be caused to be output by storing the video, suchas on an external storage device, a user device, a portable device, acloud device, and/or a portion of a partitioned memory.

Steps 710-750 may be performed for a plurality of blocks in the frame.Method 700 may be performed as a single pass video frame codingworkflow. Method 700 may be performed as a double-pass video framecoding workflow. Method 700 may be incorporated in an existing or newvideo codec standard.

For example, a decoder may receive a video from an encoder. The videomay show a children's show. The decoder may determine that the video isintended to be viewed at a distance of five to seven feet from adisplay. The decoder may determine that the video is intended to beviewed on a device with a display of twenty-four inches to fifty-fiveinches. The decoder may determine that the video is intended to beviewed in a brightly-lit environment.

Based on the viewing characteristics, the decoder may determine a numberof borders of blocks of pixels of frames of the video to be deblockingfiltered. Based on the viewing characteristics, the decoder maydetermine a deblocking filter strength for each of the blocks. Thedecoder may filter the determined number of borders of each of theblocks using the deblocking filter strengths determined for each of theblocks.

FIG. 8 shows an example method 800. At step 810, a video may bereceived. The video may be received by an encoder. The video may bereceived by a decoder. The video may comprise one or more frames. Aframe of the video may comprise a plurality of pixels. The pixels may bedivided into blocks, such as 4×4 pixel blocks or 8×8 pixel blocks.

At step 820, a content complexity of a block of pixels of a frame of thevideo may be determined. The complexity of the block may comprise acomplexity of a feature or texture shown in the block. For example, ablock showing a flat area may have a lower complexity than a blockshowing a relatively complex texture. The complexity of a block may bedetermined based on a number of details or features shown in the block.The complexity of the block may be determined based on variation ofpixels in the block. The complexity of the block may be determinedaccording to any of several methods.

At step 830, a viewing characteristic of the video may be determined.The viewing characteristic may comprise an intended distance between aviewer of the video and a display of the video. The viewingcharacteristic may comprise a resolution of the video. The viewingcharacteristic may comprise a display width of the video. The viewingcharacteristic may comprise a viewing angle of the video. The viewingcharacteristic may comprise a display pixel density of the video. Theviewing characteristic may comprise an intended and/or expected ambientillumination of a viewing setting of the video. The viewingcharacteristic may comprise a pixel dimension of a sinusoidal gratingcycle on the display of the video. The viewing characteristic maycomprise a reflection coefficient of the SDR/HDR video display. Step 830may be performed before, after, or contemporaneous to step 820.

At step 840, a number of borders of the block of pixels may bedetermined. The number of borders of the block of pixels may bedetermined based on a β threshold parameter. The β threshold parametermay be determined based on the viewing characteristic. For example, ahigher β threshold parameter may be determined for a video that isintended to be viewed at a smaller viewing distance. For example, asmaller β threshold parameter may be determined for a video that isintended to be viewed on a device with a wider display width.

Determining the β threshold parameter may comprise determining anoffset. The offset may be divided by a value. For example, the offsetmay be divided by 2. The result of the offset divided by the value maybe added to the QP value used. The result of the offset divided by thevalue added to the QP value may be used to determine β thresholdparameter, such as by determining the corresponding β thresholdparameter using the table/graph in FIG. 4. The offset may be determinedadaptively. The offset may vary according to the texture and/or contentcomplexity of one or more blocks of pixels of a frame.

At step 850, a deblocking filter strength may be determined. Thedeblocking filter strength may be determined based on a t_(c) thresholdparameter. The t_(c) threshold parameter may be determined based on theviewing characteristic. For example, a lower t_(c) threshold parametermay be determined for a video that is intended to be viewed in lowerambient lighting level.

Determining the t_(c) threshold parameter may comprise determining anoffset. The offset may be divided by a value. For example, the offsetmay be divided by 2. The result of the offset divided by the value maybe added to the QP value used. The result of the offset divided by thevalue added to the QP value may be used to determine t_(c) thresholdparameter, such as by determining the corresponding t_(c) thresholdparameter using the table/graph in FIG. 3. The offset may be determinedadaptively. The offset may vary according to the texture and/or contentcomplexity of one or more blocks of pixels of a frame.

At step 860, the number of borders may be filtered using the deblockingfilter strength. At step 870, the video may be caused to be output. Theblock having the filtered borders may be caused to be output. A framehaving the block having the filtered borders may be caused to be output.The video may be caused to be output by sending the video, such as bystreaming the video. The video may be sent to a user device, a displaydevice, a mobile device, and/or another computing device, as examples.The video may be caused to be output by displaying, such as via adisplay. The video may be caused to be output by storing the video, suchas on an external storage device, a user device, a portable device, acloud device, and/or a portion of a partitioned memory. Steps 810-860may be performed for a plurality of blocks in the frame. Method 800 maybe performed as a single pass video frame coding workflow. Method 800may be performed as a double-pass video frame coding workflow. Method800 may be incorporated in an existing or new video codec standard.

For example, an encoder may receive a video. The video may compriseframes showing a baseball game. The encoder may determine a contentcomplexity of a block of pixels of one of the frames showing a baseballtraveling mid-air. The encoder may determine a content complexity of ablock of pixels of the frame showing a portion of a ground of a baseballfield. The encoder may determine a higher content complexity of theblock of pixels showing the traveling baseball than the block of pixelsshowing the portion of the ground.

The encoder may determine that the video is intended to be viewed on adevice with a 1920×1080 pixel resolution. The encoder may determine thatthe video is intended to be viewed on a device with a 300 or 3000 pixelsper inch pixel density, for example.

Based on the content complexities and/or the viewing characteristics,the encoder may determine a number of borders of each of the frames tobe deblocking filtered. Based on the content complexities and/or theviewing characteristics, the encoder may determine a deblocking filterstrength for each of the blocks. The encoder may filter the determinednumber of borders of each of the blocks using the deblocking filterstrengths determined for each of the blocks.

FIG. 9 shows an exemplary operating environment 600 for performing thedisclosed methods. This exemplary operating environment is only anexample of an operating environment and is not intended to suggest anylimitation as to the scope of use or functionality of operatingenvironment architecture. Neither should the operating environment beinterpreted as having any dependency or requirement associated with anyone or combination of components shown in the exemplary operatingenvironment.

The present methods and systems may be operational with numerous othergeneral purpose or special purpose computing system environments orconfigurations. Examples of well-known computing systems, environments,and/or configurations that may be suitable for use with the systems andmethods comprise, but are not limited to, personal computers, servercomputers, laptop devices, and multiprocessor systems. Additionalexamples comprise set top boxes, programmable consumer electronics,network PCs, minicomputers, mainframe computers, distributed computingenvironments that comprise any of the above systems or devices, and thelike.

The processing of the disclosed methods and systems may be performed bysoftware components. The disclosed systems and methods may be describedin the general context of computer-executable instructions, such asprogram modules, being executed by one or more computers or otherdevices. Generally, program modules comprise computer code, routines,programs, objects, components, data structures, etc. that performsparticular tasks or implements particular abstract data types. Thedisclosed methods may be practiced in grid-based and distributedcomputing environments where tasks may be performed by remote processingdevices that are linked through a communications network. In adistributed computing environment, program modules may be located inboth local and remote computer storage media including memory storagedevices.

The systems and methods disclosed herein may be implemented via ageneral-purpose computing device in the form of a computing device 901.The components of the computing device 901 may comprise, but are notlimited to, one or more processors or processing units 903, a systemmemory 912, and a system bus 913 that couples various system componentsincluding the processor 903 to the system memory 912. In the case ofmultiple processing units 903, the system may utilize parallelcomputing.

The system bus 913 represents one or more of several possible types ofbus structures, including a memory bus or memory controller, aperipheral bus, an accelerated graphics port, and a processor or localbus using any of a variety of bus architectures. By way of example, sucharchitectures may comprise an Industry Standard Architecture (ISA) bus,a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, aVideo Electronics Standards Association (VESA) local bus, an AcceleratedGraphics Port (AGP) bus, and a Peripheral Component Interconnects (PCI),a PCI-Express bus, a Personal Computer Memory Card Industry Association(PCMCIA), Universal Serial Bus (USB) and the like. The bus 913, and allbuses specified in this description may be implemented over a wired orwireless network connection and each of the subsystems, including theprocessor 903, a mass storage device 904, an operating system 905,coding software 906, coding data 907, a network adapter 908, systemmemory 912, an Input/Output Interface 910, a display adapter 909, adisplay device 911, and a human machine interface 902, may be containedwithin one or more remote computing devices 914 a,b,c at physicallyseparate locations, connected through buses of this form, in effectimplementing a fully distributed system.

The computing device 901 typically comprises a variety of computerreadable media. Example readable media may be any available media thatis accessible by the computing device 901 and comprises, for example andnot meant to be limiting, both volatile and non-volatile media,removable and non-removable media. The system memory 912 comprisescomputer readable media in the form of volatile memory, such as randomaccess memory (RAM), and/or non-volatile memory, such as read onlymemory (ROM). The system memory 912 typically contains data and/orprogram modules such as operating system 905 and coding software 906that are immediately accessible to and/or are presently operated on bythe processing unit 903.

The computing device 901 may comprise other removable/non-removable,volatile/non-volatile computer storage media. By way of example, FIG. 9shows a mass storage device 904 which may provide non-volatile storageof computer code, computer readable instructions, data structures,program modules, and other data for the computing device 901. Forexample and not limitation, a mass storage device 904 may be a harddisk, a removable magnetic disk, a removable optical disk, magneticcassettes or other magnetic storage devices, flash memory cards, CD-ROM,digital versatile disks (DVD) or other optical storage, random accessmemories (RAM), read only memories (ROM), electrically erasableprogrammable read-only memory (EEPROM), and the like.

Optionally, any number of program modules may be stored on the massstorage device 904, including by way of example, an operating system 905and coding software 906. Each of the operating system 905 and codingsoftware 906 (or some combination thereof) may comprise elements of theprogramming and the coding software 906. Coding data 907 may be storedon the mass storage device 904. Coding data 907 may be stored in any ofone or more databases known in the art. Examples of such databasescomprise, DB2®, Microsoft® Access, Microsoft® SQL Server, Oracle®,mySQL, PostgreSQL, and the like. The databases may be centralized ordistributed across multiple systems.

The user may enter commands and information into the computing device901 via an input device (not shown). Examples of such input devices maycomprise, but are not limited to, a keyboard, pointing device (e.g., a“mouse”), a microphone, a joystick, a scanner, tactile input devicessuch as gloves, and other body coverings, and the like. These and otherinput devices may be connected to the processing unit 903 via a humanmachine interface 902 that is coupled to the system bus 913, but may beconnected by other interface and bus structures, such as a parallelport, game port, an Institute of Electrical and Electronics Engineers(IEEE) 994 Port (also known as a Firewire port), a serial port, or auniversal serial bus (USB).

A display device 911 may be connected to the system bus 913 via aninterface, such as a display adapter 909. It is contemplated that thecomputing device 901 may have more than one display adapter 909 and thecomputer 901 may have more than one display device 911. For example, adisplay device may comprise a monitor, a Liquid Crystal Display (LCD),or a projector. In addition to the display device 911, other outputperipheral devices may comprise components such as speakers (not shown)and a printer (not shown) which may be connected to the computing device901 via Input/Output Interface 910. Any step and/or result of themethods may be output in any form to an output device. Such output maycomprise any form of visual representation, including, but not limitedto, textual, graphical, animation, audio, tactile, and the like. Thedisplay 911 and computing device 901 may comprise part of one device, orseparate devices.

The computing device 901 may operate in a networked environment usinglogical connections to one or more remote computing devices 914 a,b,c.By way of example, a remote computing device may comprise a personalcomputer, a portable computer, a smart phone, a server, a router, anetwork computer, a peer device or other common network connectionsbetween the computing device 901 and a remote computing device 914 a,b,cmay be made via a network 915, such as a local area network (LAN) and ageneral wide area network (WAN). Such network connections may be througha network adapter 908. A network adapter 908 may be implemented in bothwired and wireless environments. Such networking environments areconventional and commonplace in dwellings, offices, enterprise-widecomputer networks, intranets, and the Internet.

Example application programs and other executable program componentssuch as the operating system 905 are shown herein as discrete blocks,although such programs and components may reside at various times indifferent storage components of the computing device 901, and may beexecuted by the data processor(s) of the computer. An implementation ofcoding software 906 may be stored on or sent across some form ofcomputer readable media. Any of the disclosed methods may be performedby computer readable instructions embodied on computer readable media.Computer readable media may comprise any available media that may beaccessed by a computer. By way of example and not limitation, computerreadable media may comprise “computer storage media” and “communicationsmedia.” “Computer storage media” comprise volatile and non-volatile,removable and non-removable media implemented in any methods ortechnology for storage of information such as computer readableinstructions, data structures, program modules, or other data. Examplecomputer storage media may comprise RAM, ROM, EEPROM, flash memory orother memory technology, CD-ROM, digital versatile disks (DVD) or otheroptical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other medium which maybe used to store the desired information and which may be accessed by acomputer.

What is claimed is:
 1. A method comprising: receiving a video;determining a content complexity of at least one block of pixels of aframe of the video based on a variation of the pixels; determining athreshold parameter, wherein determining the threshold parametercomprises determining an offset based on the content complexity;determining a viewing characteristic associated with display of thevideo to a user, wherein the viewing characteristic comprises at least acharacteristic of an intended display for viewing the video or acharacteristic of an intended setting for viewing the video;determining, based on the viewing characteristic and the thresholdparameter, a number of borders of the at least one block of pixels;determining, based on the viewing characteristic and the thresholdparameter, a deblocking filter strength; filtering, using the deblockingfilter strength, the number of borders of the at least one block ofpixels; and causing output of the filtered at least one block of pixels.2. The method of claim 1, wherein the viewing characteristic associatedwith display of the video to the user comprises at least one of anintended distance between the user of the video and a display of thevideo, a resolution of the video, a display width of the video, aviewing angle of the video, a display pixel density of the video, anambient illumination of a viewing setting of the video, a pixeldimension of a sinusoidal grating cycle on the display of the video, ora reflection coefficient of the display of the video.
 3. The method ofclaim 1, wherein the determining the deblocking filter strength is basedon a quantization parameter used to quantize the at least one block ofpixels.
 4. A device comprising: one or more processors; andinstructions, that when executed by the one or more processors, causethe device to: receive a video; determine a content complexity of atleast one block of pixels of a frame of the video based on a variationof the pixels; determine a threshold parameter, wherein determining thethreshold parameter comprises determining an offset based on the contentcomplexity; determine a viewing characteristic associated with displayof the video to a user, wherein the viewing characteristic comprises atleast a characteristic of an intended display for viewing the video or acharacteristic of an intended setting for viewing the video; determine,based on the viewing characteristic and the threshold parameter, anumber of borders of the at least one block of pixels; determine, basedon the viewing characteristic and the threshold parameter, a deblockingfilter strength; filter, using the deblocking filter strength, thenumber of borders of the at least one block of pixels; and cause outputof the filtered at least one block of pixels.
 5. The device of claim 4,wherein the viewing characteristic associated with display of the videoto the user comprises at least one of an intended distance between theuser of the video and a display of the video, a resolution of the video,a display width of the video, a viewing angle of the video, a displaypixel density of the video, an ambient illumination of a viewing settingof the video, a pixel dimension of a sinusoidal grating cycle on thedisplay of the video, or a reflection coefficient of the display of thevideo.
 6. The device of claim 4, wherein the instructions cause thedevice to determine the deblocking filter strength based on aquantization parameter used to quantize the at least one block ofpixels.
 7. A non-transitory computer-readable medium storinginstructions that, when executed, cause: receiving a video; determininga content complexity of at least one block of pixels of a frame of thevideo based on a variation of the pixels; determining a thresholdparameter, wherein determining the threshold parameter comprisesdetermining an offset based on the content complexity; determining aviewing characteristic associated with display of the video to a user,wherein the viewing characteristic comprises at least a characteristicof an intended display for viewing the video or a characteristic of anintended setting for viewing the video; determining, based on theviewing characteristic and the threshold parameter, a number of bordersof the at least one block of pixels; determining, based on the viewingcharacteristic and the threshold parameter, a deblocking filterstrength; filtering, using the deblocking filter strength, the number ofborders of the at least one block of pixels; and causing output of thefiltered at least one block of pixels.
 8. The non-transitorycomputer-readable medium of claim 7, wherein the viewing characteristicassociated with display of the video to the user comprises at least oneof an intended distance between the user of the video and a display ofthe video, a resolution of the video, a display width of the video, aviewing angle of the video, a display pixel density of the video, anambient illumination of a viewing setting of the video, a pixeldimension of a sinusoidal grating cycle on the display of the video, ora reflection coefficient of the display of the video.
 9. Thenon-transitory computer-readable medium of claim 7, wherein theinstructions cause the determining the deblocking filter strength basedon a quantization parameter used to quantize the at least one block ofpixels.
 10. The non-transitory computer-readable medium of claim 9,wherein the determined number of borders of the at least one block ofpixels is directly related to a value of the quantization parameter. 11.The non-transitory computer-readable medium of claim 7, wherein theinstructions cause the determining the deblocking filter strength basedon features shown in the block of pixels.